User-Centric Static Analysis

Every day, software developers produce complex and featurerich programs. Resulting code bases are so large that single developers cannot entirely understand them, relying on tools to help them write or debug their code. One such tool is static analysis, a method of reasoning about the runtime behavior of a program at compile time to detect bugs automatically. But from code development to reporting, to testing, different uses of static analysis have different requirements for which traditional approaches are sometimes ill-adapted. This results in more work for their users, and higher abandonment rates of otherwise powerful tools. We advocate for analyses that are centered around the user’s needs by introducing the concept of Just-in-Time (JIT) analyses. JIT analyses are adaptable to their use cases, reporting the most relevant results quickly, and computing the rest incrementally later. In particular, we explore the use case of code development with the JIT analysis Cheetah. We show in our experiments that Cheetah allows code developers to fix bugs twice as fast compared to equivalent traditional analyses. 1 PROBLEM & MOTIVATION Debugging is a critical part of software development. The earlier functional and security bugs are discovered, i.e. during design or implementation, the lower the cost to fix them is [12]. According to a study conducted by the University of Cambridge [5], programmers spend 49.9% of their time debugging. One of the primary methods used to assist them in this task is static code analysis, that is, analyzing the code of a program without running it. From Google’s Tricorder [21] to Oracle’s Parfait [8], many companies have integrated static analysis in the development phase. However, the adoption of static analysis tools still shows high abandonment rates [13]. Complex static analyses require considerable time and computational resources to return precise results, causing well-documented shortcomings. We have identified and aim to address three main challenges: • Workflow disruptions: According to the size of the analyzed code base, an analysis can take hours to complete, which makes the use of code analysis in an Integrated Development Environment (IDE) difficult. This forces programmers to separate the tasks of coding and debugging. Previous studies have shown that static analysis tools should not disrupt the developer’s workflow [7, 13], making them more likely to be adopted [24]. • False positives: Static analysis tend to report many warnings, among which many are discarded by the users [3]. • Analysis customization: Because users don’t know how an analysis operates internally, it is sometimes difficult for them to determine why certain warnings are reported, and which ones to address in priority. They pour over long lists of warnings, and put considerable effort into eliminating false positives and fixing bugs efficiently [8, 13]. While static analysis tools can be very powerful, their potential is wasted when they are not adapted to their use case. Traditional static analyses work independent of the user: users have little knowledge and control over how an analysis works internally, and the analysis does not take into account user knowledge about the analyzed code. In the case of code development, the use of static analysis can break the code developer’s workflow and add an overhead to their work. In addition, the analysis does not capitalize on the guidance the developer can provide to simplify its task (e.g. focus on certain parts of the code). In order to provide better support for debugging software -especially during code development, we advocate for more adaptive, user-centric static analyses. In our work, we explain how a large class of static analyses can be adapted to take user requirements into account during the analysis in order to return in priority those results that are most interesting to the user. Interesting results can be results that are easier to fix, faster to compute, or that are more likely to be true positives for example. In this work, we make the following contributions: • We introduce the concept of Just-in-Time (JIT) analysis [16] that allows static analysis writers to specify prioritization properties used to direct the analysis towards those results that are most interesting to the end-user. • We instantiate this concept through a layered analysis system, and show that existing static analyses can be adapted to support it with minimal changes. • We implement Cheetah, a JIT analysis that detects privacy leaks in Android applications. Focusing on the use case of code development, Cheetah returns in priority those results that are located around the code developer’s working set, gradually expanding the analysis scope to encompass methods, classes, and modules further away. Early computations produce simple results quickly enough for Cheetah to be integrated in an IDE. • We evaluate Cheetah, focusing on performance and developer experience. Our experiments show that Cheetah is able to return initial results under a second, and that using Cheetah in the IDE enables code developers to fix warnings twice as fast as with a traditional analysis. We refer the reader to our technical report [6] for the JIT algorithm, its proofs of soundness and termination, the details of Cheetah’s implementation, and the setup and raw data of our empirical evaluation and user study. Cheetah is open-sourced [6] and available under the EPL license.